Data led growth

Problem statement

Bids per vehicle needs to grow for SPEEDonline, an auction website for salvage vehicles. It has been steadily declining from 6 in FY22 to 4 in FY24, in turn decreasing the average highest bid value, leading to higher settlement cost for insurers.

Overview

This document outlines the top 3 experiments that can be conducted to improve bids per vehicle for all vehicles without making major changes to SPEEDonline.

Constraints

• We continue to run only blind or silent auctions, where the bidder or buyer doesn’t see the current highest bid value
• There is currently no way to automatically refresh the screen with new incoming auctions due the legacy nature of the web app
• Mobile experience is poor, cannot be transformed without a complete re-write of the website

Observations

• Only 75% vehicles receive any bids
• Auction duration is impacted by the time it takes for internal teams to pre-process incoming auction data manually and publish it when ready.

Hypothesis

• IF we allowed the user to add a filter called “Vehicles with no bids”, allowing buyers to easily identify opportunities to win vehicles cheaply, THEN the % vehicles with bids will increase, in turn improving bids per vehicle.
• IF we highlighted a select set of vehicles as “Direct contact” to indicate that we will facilitate the sale between vehicle owner and buyer, in alignment with insurers, THEN the % vehicles with bids will increase, in turn improving bids per vehicle.

Goal

Running these experiments with the objective of improving bids per vehicle is more cost-effective than optimizing pre-processing time or revamping the website completely in hope of better UX leading to higher bids per vehicle. The more the bids, the higher the avg bid value, the lesser the settlement cost for the insurer, the lower payout the owner gets, the more inclined they will be to sell the damaged vehicle to our buyer - leading to a healthy sales conversion rate and improving our revenue from buyers, while ensuring that the insurers continue to send us high volumes of vehicles to auction.

Success Metrics

• Worst case scenario : Absolute increase in % vehicles with bids by 1%
• Best case scenario : Absolute increase in % vehicles with bids by 5%

Experiment design

Variation design

Vehicles without bids

• Variant - A : A new filter value called “Vehicles without bids” will be added to the side bar filters for buyers to easily identify vehicles without bids, see if the vehicle interests them and place bids quickly.

Direct contact

• Variant - B : A purple call icon is added to each of the auction cards and thumbnail photo of the auctions that qualify for the Direct contact model.

• A new featured filter called “Direct contact” is also added for buyers to easily filter such vehicles.

Control version

Audience & Sample size

Segment

Out of 5000+ auctions per day in Germany

Control : 2000 

Variant A : 1500

Variant B : 1500

Implementation and A/A test

We didn’t have a chance to run the A/A test due to time constraints. Idea was already validated during user surveys so we went ahead with the implementation.

Post experiment

Experiment result

For both variants, there is no change in pricing, discounts for buyers or vendors. However, there is a cost overhead associated with “Direct contact” because it involves coordination  between owner and buyer with additional manual effort. 

• Variant-A : Vehicles without bids filter - Worked very well. 
• • Expected increase in % vehicles without bids : 1-5%
  • Actual increase : 10%
  • Statistical significance (p value) : 0.03
• Variant-B : Direct contact - worked only for vehicles coming from reputed insurers
• • Expected increase in % vehicles without bids : 1-5%
  • Actual increase : 1%
  • Statistical significance (p value) : 0.15

Release decision

At the end, we decided to 

• Scale the “vehicles with no bids” filter to all vehicles
• Enable “direct contact” only for leading insurers and only for 10% of vehicles to also manage the manual overhead in post-auction operations cost-effectively.

Learnings

• Understanding bidding activity at a granular level is key before designing experiments. 
• Important to fully align on experiments with all functions involved (E.g. operations) to avoid last minute surprises on data and resource availability.